AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.687821	2.41
	2_DecisionTree	Decision Tree	logloss	0.431098	17.75
the best	3_Default_Xgboost	Xgboost	logloss	0.244482	33.84
	4_Default_NeuralNetwork	Neural Network	logloss	0.376986	10.94
	5_Default_RandomForest	Random Forest	logloss	0.375043	21.16
	Ensemble	Ensemble	logloss	0.244482	1.55

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	5

Metric details

	score	threshold
logloss	0.244482	nan
auc	0.963344	nan
f1	0.912928	0.520228
accuracy	0.90389	0.520228
precision	1	0.999119
recall	1	5.80332e-05
mcc	0.805687	0.520228

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.244482	nan
auc	0.963344	nan
f1	0.912928	0.520228
accuracy	0.90389	0.520228
precision	0.912367	0.520228
recall	0.913489	0.520228
mcc	0.805687	0.520228

Confusion matrix (at threshold=0.520228)

	Predicted as long	Predicted as short
Labeled as long	1769	214
Labeled as short	211	2228

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

19.6 seconds

Metric details

	score	threshold
logloss	0.375043	nan
auc	0.917267	nan
f1	0.860438	0.39563
accuracy	0.837178	0.491837
precision	0.995935	0.965178
recall	1	0.0481873
mcc	0.673689	0.600846

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.375043	nan
auc	0.917267	nan
f1	0.854487	0.491837
accuracy	0.837178	0.491837
precision	0.842567	0.491837
recall	0.866749	0.491837
mcc	0.670111	0.491837

Confusion matrix (at threshold=0.491837)

	Predicted as long	Predicted as short
Labeled as long	1588	395
Labeled as short	325	2114

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.4 seconds

Metric details

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.687821	nan
auc	0.5	nan
f1	0.710975	0.496442
accuracy	0.55156	0.496442
precision	0.55156	0.496442
recall	1	0.496442
mcc	0	0.496442

Confusion matrix (at threshold=0.496442)

	Predicted as long	Predicted as short
Labeled as long	0	1983
Labeled as short	0	2439

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

32.3 seconds

Metric details

	score	threshold
logloss	0.244482	nan
auc	0.963344	nan
f1	0.912928	0.520228
accuracy	0.90389	0.520228
precision	1	0.999119
recall	1	5.80332e-05
mcc	0.805687	0.520228

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.244482	nan
auc	0.963344	nan
f1	0.912928	0.520228
accuracy	0.90389	0.520228
precision	0.912367	0.520228
recall	0.913489	0.520228
mcc	0.805687	0.520228

Confusion matrix (at threshold=0.520228)

	Predicted as long	Predicted as short
Labeled as long	1769	214
Labeled as short	211	2228

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

16.3 seconds

Metric details

	score	threshold
logloss	0.431098	nan
auc	0.872229	nan
f1	0.833564	0.504445
accuracy	0.809362	0.504445
precision	0.960432	0.829987
recall	1	0.0679791
mcc	0.613485	0.504445

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.431098	nan
auc	0.872229	nan
f1	0.833564	0.504445
accuracy	0.809362	0.504445
precision	0.803884	0.504445
recall	0.865519	0.504445
mcc	0.613485	0.504445

Confusion matrix (at threshold=0.504445)

	Predicted as long	Predicted as short
Labeled as long	1468	515
Labeled as short	328	2111

Learning curves

Decision Tree

Tree #1

Rules

if (Total Costs <= 9081.005) and (Total Costs <= 4890.975) and (CCS Diagnosis Code <= 253.5) then class: short (proba: 94.79%) | based on 3,260 samples

if (Total Costs <= 9081.005) and (Total Costs > 4890.975) and (CCS Diagnosis Code <= 253.5) then class: short (proba: 71.21%) | based on 3,102 samples

if (Total Costs > 9081.005) and (Type of Admission > 0.5) and (Total Charges > 45902.932) then class: long (proba: 92.45%) | based on 2,489 samples

if (Total Costs > 9081.005) and (Type of Admission > 0.5) and (Total Charges <= 45902.932) then class: long (proba: 69.89%) | based on 2,039 samples

if (Total Costs > 9081.005) and (Type of Admission <= 0.5) and (Total Costs <= 24177.08) then class: short (proba: 65.45%) | based on 1,094 samples

if (Total Costs > 9081.005) and (Type of Admission <= 0.5) and (Total Costs > 24177.08) then class: long (proba: 74.7%) | based on 494 samples

if (Total Costs <= 9081.005) and (Total Costs <= 4890.975) and (CCS Diagnosis Code > 253.5) then class: short (proba: 70.78%) | based on 438 samples

if (Total Costs <= 9081.005) and (Total Costs > 4890.975) and (CCS Diagnosis Code > 253.5) then class: long (proba: 81.38%) | based on 349 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

9.7 seconds

Metric details

	score	threshold
logloss	0.376986	nan
auc	0.926112	nan
f1	0.872605	0.448384
accuracy	0.857983	0.45332
precision	0.982405	0.983634
recall	1	5.15408e-18
mcc	0.713556	0.486738

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.376986	nan
auc	0.926112	nan
f1	0.871153	0.45332
accuracy	0.857983	0.45332
precision	0.871869	0.45332
recall	0.870439	0.45332
mcc	0.712968	0.45332

Confusion matrix (at threshold=0.45332)

	Predicted as long	Predicted as short
Labeled as long	1671	312
Labeled as short	316	2123

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back